The Evaluation of Similarity Metrics in Collaborative Filtering Recommenders

Collaborative filtering recommenders predict user opinions regardless of item content. Memory-based implementations generate recommendations on demand, without any pre-computation. Even though such algorithms are slow, they are known for their superior personalization. We study collaborative filtering recommendations on the Netflix dataset. It is essentially a set of (user id, item id, rating) tuples that fit perfectly into a relational database model, with standard data manipulation operations. The simple structure of our system allows consistent and repeatable measurements. Additionally, it natively supports multi-threaded processing, which helps address the inherent performance drawbacks of our approach. We evaluate multiple similarity measures in a traditional collaborative filtering process. We also consider combinations of complementary measures, especially in edge cases when one of them falls short, e.g., a user with uniform ratings. We examine prediction accuracy, classification accuracy, confusion statistics, and actual/predicted distribution compatibility to find the best way to quantify vector similarity.